Generalized Nonparametric Mixed-Effect Models: Computation and Smoothing Parameter Selection

Generalized linear mixed-effect models are widely used for the analysis of correlated nonGaussian data such as those found in longitudinal studies. In this article, we consider extensions with nonparametric fixed effects and parametric random effects. The estimation is through the penalized likelihood method, and our focus is on the efficient computation and the effective smoothing parameter selection. To assist efficient computation, the joint likelihood of the observations and the latent variables of the random effects is used instead of the marginal likelihood of the observations. For the selection of smoothing parameters and correlation parameters, direct cross-validation techniques are employed; the effectiveness of cross-validation with respect to a few loss functions are evaluated through simulation studies. Real data examples are presented to illustrate potential applications of the methodology. Open-source R code is demonstrated in an appendix.